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Evolutionary social dilemma games are extended by an additional matching-pennies game that modifies the 
collected payoffs. In a spatial version players are distributed on a square lattice and interact with their neigh¬ 
bors. Firstly, we show that the matching-pennies game can be considered as the microscopic force of the Red 
Queen effect that breaks the detailed balance and induces eddies in the microscopic probability currents if the 
strategy update is analogous to the Glauber dynamics for the kinetic Ising models. The resulting loops in prob¬ 
ability current breaks symmetry between the chessboard-like arrangements of strategies via a bottleneck effect 
occurring along the four-edge loops in the microscopic states. The impact of this congestion is analogous to 
the application of a staggered magnetic field in the Ising model, that is, the order-disorder critical transition is 
wiped out by noise. It is illustrated that the congestion induced symmetry breaking can be beneficial for the 
whole community within a certain region of parameters. 

PACS numbers: 89.65.-s, 89.75.Fb, 87.23.-n, 05.70.Ln 


I. INTRODUCTION 

Multi-agent evolutionary games on networks M are fre¬ 
quently used approach to study evolutionary processes charac¬ 
teristic to biological and social systems. In these models play¬ 
ers are located on the sites of a network (or lattice) and their 
strategies represent biological entities or individual choices. 
For most of the cases the pair interactions among players are 
characterized by two-player games il. 

In these evolutionary games a uniform, symmetric, two- 
^ayer, two-strategy game defines the payoffs for both players 
1^. In that simple cases we can distinguish only four possible 
strategy pairs (henceforth strategy profiles) for the elementary 
interactions. In Fig.[T]these strategy profiles are illustrated by 



FIG. 1: Flow diagrams for two-payer two-strategy games. The left 
and right diagrams show the payoffs for the prisoner’s dilemmas 
(S < 0 and T > 1) and matching-pennies game. Arrows along 
the edges point towards the preferred strategy profile from the view 
point of player who modifies unilaterally her strategy. 

large boxes with white and black bullets representing the two 
possible strategies for each player. The actual payoffs are de¬ 
noted within these bullets. For the prisoner’s dilemma, as the 


most challenging social dilemmas, the black and white bullets 
refer to Defection {D) and Cooperation (C) strategies while 
for the matching-pennies game these symbols denote Head 
and Tail (two sides of a coin) states. Henceforth we use the 
abbreviated nomination of C and D strategies and the tradi¬ 
tional notation o^^offs introduced for the investigation of 
social dilemmas IQIMI- 

The mentioned games represent two significantly different 
behaviors as it is illustrated in the flow diagrams where the 
edges connect those strategy profiles that can be transformed 
into each other by a single strategy change of a player. For the 
symmetric two-player, two-strategy games the payoff varia¬ 
tions of the active player from both the DD and CC strategy 
profiles are equivalent. In fact, this is the reason why the sum 
of the payoff variations of active players along the single loop 
is zero independently of the values of S and T. Consequently, 
all the symmetric two-player two-strategy games are potential 
games For the potential games one can And a poten¬ 

tial (as a function of strategy profile) summarizing the driving 
force for the unilateral strategy changes on the analogy of the 
potential energy in physical systems. For the multi-strategy 
systems potential can exist if the above condition is satisfied 
along all the possible loops in the space of strategy profiles. 
The existence of potential is accompanied with some general 
features. Namely, the maximal value of potential is achieved 
at a pure Nash equilibrium; additional pure Nash equilibria 
are represented by nodes without outgoing edges; and the ab¬ 
sence of directed loops are forbidden in the flow diagram. For 
the potential games the latter feature results in that one can 
arrive to one of the pure Nash equilibria when following the 
direction of arrows from any initial state for the multi-strategy 
and/or multi-agent games 113, [13 • 

If a multi-agent system is built up from two-player potential 
games then the potential for the whole system will be the sum 
of the pair potentials. Blume ifldliTSl] has shown that the evo- 
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lutionary potential games evolve into a Gibbs ensemble if the 
evolution is controlled by a l ogit rule ifl^ that is similar to the 
Glauber 10 or Metropolis ifl^ dynamics introduced for the 


investigation of kinetic Ising models. In other words, all these 
multi-agent systems with symmetric two-strategy interactions 
are equivalent to an Ising model 

On the contrary, the flow diagram of matching-pennies 
game represents a fundamentally different interaction. In that 
case the players are not equivalent and one of the players is 
always motivated to change her strategy. As a result, there is 
a uniform payoff increase (driving force) along the four edges 
that represents a directed loop in the flow diagram. For this 
game potential does not exist and the single Nash equilibrium 
is a mixed strategy profile where both players choose their 
strategy at random 12^12^ . The matching-pennies game can 
be interpreted as a driving force creating circular transitions in 
the flow diagram and probability current loops in the dynam¬ 
ical graphs that can be quantified by evaluating the entropy 
production MM- The relevance of a conceptually similar 
circular transitions in biological systems was first described 
by van Valen ll^ 1^ who named this effect as Red Queen 
mechanism. Similar endless races can be observed in social 
systems between buyers and seller s 112^ . p roperty owners and 
crimin als [1^ . males and females 11^ 1^ . or conformists and 
rebels Bit] . The consideration of an evolutionary multi-agent 
system on network with this type of interactions prescribes 
two kinds of players distributed on bipartite graphs or lattices 


The evolutionary matching-pennies game and its combina¬ 
tion with an anti-coordination game were considered previ¬ 
ously for several structured populations 1I2II] . In the present 
paper we extend these analysis to systems where the men¬ 
tioned probability current loops are distorted by bottleneck 
effects which can modify the macroscopic behavior signifi¬ 
cantly. 


II. THE MODEL 

Now we study spatial evolutionary games with players lo¬ 
cated on a square lattice with periodic boundary conditions. 
Each player has two strategies and collects income with play¬ 
ing games with the four nearest neighbors. The pair interac¬ 
tion will be composed of a symmetric two-player two-strategy 
game (a social dilemma with T — S parametrization) and 
a matching-pennies game. Our analysis will focus onto the 
region of hawk-dove games (T > 1 and S > 0) where 
chessboard-like strategy arrangements occur for myopic or 
Glaube r typ e strategy updates preferred in statistical physics 

illlllll. 

Both the suitable description of the chessboard-like strat¬ 
egy arrangements and the application of the matchingpennies 
game require the division of the square lattice into two equiv¬ 
alent sublattices denoted a = X orY resembling the black 
and white squares on a chessboard. For this sublattice division 
each site x (x G X) is surrounded by four nearest neighbors 
belonging to the opposite sublattice (x + S G Y) and vice 
versa. It is convenient to denote the strategies of players at 


sites X and y by unite vectors as 


= D = 



, or 



( 1 ) 


Using this traditional notation the player’s income can be 
given by a sum of matrix products 

Ux - ^ ^ ■ ASa;-|-(5 5 Uy - ^ ^ Sy • BSy _|_(5 , (2) 

S <5 


where the summation runs over the four nearest neighbor sites 
belonging to the opposite sublattice. According to the model 
definition the pair interactions of nearest neighbors are the 
composition of a social dilemma and a cyclic game. The cor¬ 
responding payoff matrix is given as: 


A = 


e T — e 
S — e 1 + e 



T + e 
1 — e 


(3) 


where e quantifies the strength of matching-pennies compo¬ 
nent. 

During the elementary step of the evolutionary process a 
player is chosen randomly and she modifies her strategy from 
Sa; to with a probability depending on the payoff difference 
— Ux) between the final and initial states. Namely, the 
strategy update probability is given as 


W{Sx —>■ s^, S_a;) 


1 

1 -f exp[(Ma; - Ux)/K] ’ 


(4) 


where the strategy profile for the rest of players, denoted tra¬ 
ditionally as S-x, remains unchanged. 

This transition favors the state providing higher payoff for 
the player x and similar rule is applied for the players of the 
opposite sublattice. Notice that W{Sx s_ 3 ;) ~ 1 (or 0) 

if Ux <SC u'x (or Ux ^ u'x) while the ’’width” of the transient 
region is proportional to K that measures the magnitude of 
noise or the strength of selection. Noteworthy that the prob¬ 
ability of strategy change [given by Eq. ([Hi] is equivalent to 
those suggested by Glauber ifl^ for the kinetic Ising model. 

For e = 0 the above system satisfies the condition of poten¬ 
tial games because the sum of payoff variation along the 
single loop is zero, as illustrated in Fig. [T] In fact the kinetic 
Ising model and the corresponding two-strategy evolutionary 
potential games are equivalent, thus the coupling constant (J) 
between the neighboring spins (sx = ±1) and the strength (h) 
of the external magnetic field can be expressed by the payoff 
parameters as J = (1 — 5" — T)/4 and h = 1 + S — T while K 
corre^onds to the temperature in the thermodynamical sys¬ 
tem 10 [Ill, [T^ |2^ . The effect of e on the transition from the 
anti-ferromagnetic to the paramagnetic phase as a function of 
K is investigated in a recent paper for the absence of external 
field h it^ . It is found that the disturbance of the matching¬ 
pennies component reduces the critical value of K while the 
universal (Ising type) features of the critical transition are pre¬ 
served. 

In the present work we explore how the stationary states 
of an evolutionary social dilemmas are influenced by the 
matching-pennies game in the presence of an external field 
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{h ^ 0) representin g ga mes with S ^ T — 1. In the litera¬ 
ture of Ising models 11^ it is well described that at zero tem¬ 
perature the ordered anti-ferromagnetic state is transformed 
into the preferred ferromagnetic state if h exceeds a threshold 
value proportional to J (the pre-factor depends on the num¬ 
ber neighbors determined by the lattice structure). On the 
other hand, by increasing temperature K the system exhibits 
an Ising type critical phase transition from the ordered anti¬ 
ferromagnetic state to the disordered spin arrangement at a 
critical point (Kc) decreasing with \h\ Now it will 

be shown that the latter robust critical transition is smoothed 
out for |e| > 0. For this purpose we first study a two-player 
game exhibiting the congestion phenomenon that results in 
macroscopic changes in the multi-player systems. 


III. CONGESTION FOR TWO-PLAYER SYSTEM 

In this section we study a two player game with the same 
dynamical rule introduced in the previous section. In that case 
the system has only four microscopic states [s = {D,D), 
{D, C), {C, D), or {C, C)] and the corresponding probabili¬ 
ties p(s) in the stationary state can be determined numerically 
by solving the suitable equations of motion for any values of 
payoff parameters and noise levels. In the absence of neigh¬ 
borhood, we can apply a simplified version of the traditional 
pair approximation (for a brief survey see fiHH]). Figure 
|2]compares the configuration probabilities in dependence of e 
for the all the four possible strategy pairs at a fixed noise level. 
The dashed lines in the left panel of Fig. |2] represent the be- 



FIG. 2: Probabilities of strategy pairs as a function of e for T = 1.5, 
S = 0.5 (left panel, dashed lines) and T — 1.4, S — 0.3 (right 
panel, solid lines) at K = 0.3. 

havior when the {D, D) and (C, C) strategy pairs appear with 
the same probability for arbitrary values of e. This is equiva¬ 
lent to /i = 0 case for the terminology of Ising model. Here 
p(Zl, C) = p{C^ D) for any values of e and all the four con¬ 
figuration probabilities tend to 1/4 if the matching-pennies 
components dominate the game, as it is discussed in ll2ll] . 


The latter behavior reflects the fact that the matching-pennies 
game has only one mixed Nash equilibrium when both players 
choose their strategies at random. Notice furthermore that the 
variations in the configuration probabilities are even functions 
of s. 

Significantly different behavior can be observed ifl + S” — 
T = h ^ 0 even for e = 0. The right plot of Fig. |2] shows 
a situation when the preferred (C, D) and {D, C) strategies 
are present with the same high probability while p{D, D) > 
p{C, C) that is obtained for the payoff parameters given in 
the caption of Fig. |2] It is emphasized that in the absence 
of matching-pennies component the system satisfies the con¬ 
dition of detailed balance, that is, the forward and backward 
transitions appear with the same probabilities along the four 
edges of the flow diagram as it is illustrated in the left plot of 

Fig. El 



FIG. 3: Configuration probabilities are illustrated by the height 
of columns for the absence (left plot) and presence (right plot) of 
matching-pennies component for T — 1.4 and S — 0.3. In the 
right plot the directed ellipse illustrates the emergence of a proba¬ 
bility current loop that modifies the configuration probabilities via a 
bottleneck effect and the white arrows on the faces of columns show 
how the probabilities change due to the congestion phenomenon. 

The presence of the matching-pennies component, how¬ 
ever, breaks the detailed balance and induces a probability 
current through the four-edge loop with a strength character¬ 
ized by the difference of transition frequencies between the 
forward and backward directions along the four edges. In 
the steady state these probab ility currents should be equiv¬ 
alent due to Kirchhoff laws 1401] . This uniform probability 
current modifies the configuration probabilities p(s) and de¬ 
stroys the equivalence between {D, C) and (C, D) configura¬ 
tions because the {D, D) and {C, C) states represent different 
barriers (widths of bottleneck) for the probability current flow. 
As a result, p{D, C) {p{C, D)} increases [decreases] linearly 
with with the strength of the matching-pennies component for 
small values of e. The white arrows in Fig. E] show that at the 
bottleneck states the corresponding probabilities increase as it 
happens before the narrower bottleneck while the probability 
of the fourth should be reduced because p{s) = 1. 

Notice that the linear dependence on e implies that the be¬ 
haviors of p{D, C) and p{C^ D) are exchanged when revers¬ 
ing the sign of e, that is accompanied with a reversal of prob¬ 
ability current, too. For high values of £ the system tends 
towards a uniform strategy distribution as mentioned above. 
Consequently, if e is increased then p{C, D) exhibits a local 
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maximum at e ~ 0.1 if T = 1.4, S' = 0.3, and K = 0.3. For 
the opposite sign of e the variations of p{D, C) and p{C, D) 
are exchanged as shown in Fig. |2] The results of this con¬ 
gestion phenomenon is resembling the impact of a staggered 
magnetic field hs {hg is positive for the sites of sublattice X 
and negative on the sublattice Y) for the anti-ferromagnetic 
Ising models that prefers one of the ordered arrangements to 
the other. 

The driving effect of the matching-pennies component 
seems to be similar to the effect of external electric field in 
driven lattice gas models For these models the exter¬ 

nal field drives the system out off the Boltzmann distribution 
and generates a permanent particle transport when peri¬ 
odic boundary conditions are applied. It breaks inherent sym¬ 
metries 1 I 43 I] . enhances the effect of noise and the impact of 
local inhomogeneities ll44|] . In case of repulsive interaction, 
where similar checkerboard-like ordered phase is formed in 
the half-filled lattice ll^ld^ , the order-disorder phase transi¬ 
tion is modified due to the particle transport Lastly we 
note that the effect of bottlenecks along a ring was recently 
studied by considering the asymmetric exclusion process 1481] 
that is the simplest version of the driven lattice gases. The 
latter investigation is motivated by biological experiments in¬ 
dicating unidirectional circular ribosome translocations along 
messenger RNA loops with defects or slow codons in cells 

iS. 

The application of the matching-pennies component in 
evolutionary games induces local probability currents within 
many significantly shorter loops that can be quantified even 
for two neighboring players when considering the transitions 
among the four possible strategy profiles for most of their 
quenched neighborhood ll^ . 


IV. CONGESTION FOR MULTI-PLAYER SYSTEM 

The above-mentioned weak effect of the matching-pennies 
component remains valid for each pair and it is amplified for 
the multi-agent games. As a result the macroscopic system 
will evolve into the preferred ordered strategy distribution at 
sufficiently low noise levels. This fact is illustrated in Fig. 0] 
where the frequency of strategy C {pa{C)) is plotted as a 
function of noise level K for both sublattices (a = X, Y). 
For e = 0 this system shows an Ising type critical transition 
at Kc — 0.3922(1) from the sublattice ordered strategy ar¬ 
rangement to a disordered state 1^ . This critical transition, 
however, is smoothed out if the game includes the matching- 
pennies component. 

Open symbols in Fig.|4]show the Monte Carlo (MC) results 
we obtained for £ = 0.1 on square lattice with a linear size 
L = 400. In that cases the sampling and relaxation times 
are chosen to be tg = tr = 10, 000 MCS where within the 
time unit (MCS) each player has a chance once on average to 
modify her strategy. When reversing the sign of e then the 
strategy occupations in the sublattices are exchanged. As we 
argued, this phenomenon is a straightforward consequence of 
the congestion effect we described in the two-player model. 
Furthermore, the sublattice occupations tends toward the pre- 
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FIG. 4: Frequency of strategy C as a function of K in sublattice X 
(open diamonds) and Y (open boxes) for T = 1.4, S = 0.3, and 
£ = 0.1. The filled symbols indicate the same quantities for £ = 0. 

diction of Ising model if £ 0 in a way resembling the van¬ 

ishing of the staggered magnetic field. Evidently, we have to 
use significantly larger system sizes (L — 2000) and longer 
runs {tg = tr = 10® MCS) for obtaining the reference data at 
£ = 0 in order to suppress the undesired effects of the diverg¬ 
ing fluctuations and critical slowing down when approaching 
the above-mentioned critical point. 

Our numerical analysis is repeated in the full payoff param¬ 
eter region, that covers all social dilemma games at fixed val¬ 
ues of £ and noise level (K = 0.3). The MC data are summed 
in Fig. |5] where the lines are obtained by varying T — S' at 
fixed values of T -F S. The MC results indicate the absence of 



FIG. 5: (color online) Frequency of strategy C as a function of payoff 
parameters T and S in both sublattices for K = 0.3 and e = 0.1. 
Dashed (red and blue) lines illustrate MC data where the sublattices 
are equivalent. The solid (purple) and dotted (green) lines show pc 
in the sublattices X and Y. 

sublattice ordering, that is px{C) = py{C), within a large 
regions of the prisoner’s dilemma, harmony, and stag hunt 
games. The C strategies dominate equally both sublattices 
in the zero noise limit if T < 1 and (T — 1 — S') < 0 and the 
homogeneous D state is stable if S < 0 and (T — 1 — S) >0 
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for the zero noise limit (K —?> 0). 

Staying at the same limit only one of the sublattice ordered 
structures appears in the hawk-dove game quadrant (T > 1 
and S > 0). More precisely, px{C) 1 and pyiC) 0 
if e > 0 and [T — 1 — S) < 0 or when e < 0 and (T — 
1 — S') > 0. Evidently, when reversing the sign of e and 
(T — 1 — S) separately then the preference between the two 
sublattice ordered structures are exchanged. 

The similarity between the impact of staggered magnetic 
field in the Ising model and the introduction of a matching- 
pennies component in the hawk-dove game can be demon¬ 
strated by monitoring the motion of an interface separating 
the two sublattice ordered arrangements. For both cases the 
visualization of the evolutionary process shows clearly the ex¬ 
pansion of the preferred sublattice ordered structure via the 
motion of separating interface. 

It is emphasized that the two-fold degeneracy can be ob¬ 
served only along the line T—1 — S = h — Q separating re¬ 
gions where the chessboard and anti-chessboard arrangements 
of the D and C strategies occur as it is shown in Fig. |3 Note¬ 
worthy, that Ising type critical transitions can take place only 
along this line if the noise level is increased. 

When decreasing the values of e and K then the widths of 
the intermediate transition regions shrink, too. Additionally, 
one can observe a disordered state at close vicinity of the cen¬ 
tral point (T — 1 = 5” = 0) of the T — S plane where both 
the coupling constant J and the external magnetic field h van¬ 
ish in the equivalent Ising model. Evidently, the absence of 
ordering will also characterize the system behavior if e and/or 
K exceed threshold values depending on T and S. 


V. EXPLOITATION OF CONGESTION 

The systematic analysis of the evolutionary games has high¬ 
lighted the existence of many different mechanisms enhancing 
the total income of societies even for the cases of social dilem¬ 
mas (for a survey see 00.11^ Bill)- It is akeady known that 
the total income of society can be influenced by the game it¬ 
self (payoffs and strategy set), the connectivity network, the 
dynamical rules including the noise level and allowing co¬ 
evolutionary processes in all ingredients of the systems. In the 
light of these results the utilization of congestion phenomenon 
emerges directly. The relevance of this question is stressed by 
recent experiments investigating human behavior in real-life 
situations involving the matching-pennies component in the 
payoffs ilmill. 

The above results justify that the main impact of matching- 
pennies is related to the preference of one of the chessboard¬ 
like strategy arrangements that suppresses the critical transi¬ 
tion related to the existence of two equivalent (optimal) or¬ 
dered strategy arrangements. As a result, the preferred struc¬ 
ture dominates the system behavior if K > Kc within the 
region of hawk-dove games. The latter effect can increase the 
total income because of the reduction of the length of inter¬ 
faces responsible to the loss in the total payoff. Conceptually 
similar congestion induced increase of collective income may 
be expected in human societies. 


For an illustration of the mentioned phenomenon we com¬ 
pare the average payoffs as a function of noise K for three 
different evolutionary rules whereas the payoff parameters (T 
and S) are fixed. For the selected payoffs (T = 1.9 and 
S = 0.3) the maximum average payoff [2{T -f 5")] is achieved 
if the C and D strategies form a chessboard-like structure 
in the zero noise limit. It is emphasized that the matching- 
pennies component does not affect the average payoff as it is 
a zero-sum game. 

Figure |6] illustrates that the presence of matching-pennies 
component increases the average payoff for the given payoff 
parameters. The most relevant increase occurs in the vicin¬ 
ity of the critical noise level [KciT = 1.9, S' = 0.3) = 
0.590(4)]. The importance of congestion phenomenon be- 



K 

FIG. 6: Comparison of average payoffs versus noise (K) for evo¬ 
lutionary games on square lattice at T = 1.9 and S — 0.3. Open 
circles and boxes represent MC data obtained for e = 0 and 0.1 if 
the evolution is controlled by the logit rule defined by Eq. lEll. Solid 
line illustrates MC results obtained when the evolution is controlled 
by collective pairwise strategy update (0|. 

comes more striking when the average payoff is compared to 
results obtained for other types of dynamical rules. Within 
the hawk-dove region the imitation-based rules can not be 
considered as an adequate reference because the imitation of 
the neighboring strate gy p revents the formation of the optimal 
strategy arrangement 11^ . Up to now one of the highest av¬ 
erage payoffs is achieved ^ the application of the collective 
pair-wise strategy update 1541] where the stochasticity is intro¬ 
duced via a noise parameter K, too. For the latter rule the fra¬ 
ternal players take into consideration the co-player’s income. 
The corresponding MC data of the average payoffs are illus¬ 
trated by the solid line in Fig. |6] Accordingly, there is a region 
of K where the introduction of the matching-pennies payoffs 
provide the highest average income for the whole population. 

VI. SUMMARY 

In this paper we have studied two-strategy evolutionary 
games on square lattice when matching-pennies games with a 
strength e are added to the payoffs in order to modify the inter¬ 
actions between neighboring players. Here matching-pennies 
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game represents the simplest cyclic dominant game for the 
description of Red Queen effect in two-strategy games if the 
evolutionary process is controlled by the logit rule resembling 
the Glauber dynamics in statistical physics. In that case the 
matching-pennies component of the pair interactions can be 
considered as a microscopic driving force that destroys the de¬ 
tailed balance by inducing probability current loops through¬ 
out the four microscopic states for each pair interactions. 

The presence of the matching-pennies component elimi¬ 
nates the equivalence between the players residing in the two 
sublattices. Additionally, the equivalence between the two or¬ 
dered (chessboard- and anti-chessboard-like) strategy distri¬ 
butions is also destroyed via a contagion mechanism affect¬ 
ing the configuration probabilities if the disfavored strategy 
pairs [e.g. {D, D) and (C, C)] would be present with different 
probabilities at e = 0. The direction of preference varies with 
the sign of e. Note that the presence of both ordered phases 
with equal weights is disadvantageous because along the sep¬ 
arating interfaces frustrating players cannot enjoy sufficiently 
high payoff. This ambiguity can be resolved by matching- 
pennies component in a desired way. 


Finally we emphasize that similar cyclic dominance can 
be created by the rock-paper-scissors component in three- 
strategy evolutionary games for equivalent players as it is re¬ 
ported for the description of many systems (for a survey see 
refs, m0,0.[nil)- In the latter cases, however, the matching- 
pennies components and their effects can be recognized within 
all the 2x2 sub-games where the interacting players are lim¬ 
ited to use two different strategy pairs. The latter feature un¬ 
derlines the importance of the matching pennies games as it 
can be included within the 2x2 sub-games for most of the 
symmetric n x n(n > 2) matrix games. 
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